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1.  INTRODUCTION 


A  stationary  first-order  autoregressive  process  with  exponential  marginal 
distributions  was  defined  In  Gaver  and  Lewis  (Ref.  1).  The  process  is  defined 
by  the  following  recursion: 


X  -  pX  .  +  I  E  where  0  <  p  <1  (1) 

n  n-i  n  n 


and  I  -  0  with  probability  p,  I ■  1  with  probability  1  -  p,  and  {E  }  is  an 

independent  identically  distributed  (l.i.d.)  sequence  of  exponential  random 

variables  with  rate  parameter  X.  The  {i  }  sequence  is  also  l.i.d.  and  is 

n 

independent  of  {XQ}« 

The  sequence  is  a  special  case  of  the  EARMA( 1,1)  sequence  studied  in 
Jacobs  and  Lewis  (Ref.  2).  Properties  of  the  sequence  are  given  in  Gaver  and 
Lewis  (Ref.  1),  Jacobs  and  Lewis  (Ref.  2),  and  Chernlck  (Ref.  3).  A  relation¬ 
ship  between  the  sequence  (which  is  denoted  by  EAR(l))  and  another  Markovian 
exponential  process  (Tavares  (Ref.  4))  is  pointed  out  in  Chernlck,  et.  al. 
(Ref.  5). 


In  section  5  of  Gaver  and  Lewis  (Ref.  1)  it  is  observed  that  when 

p  >  0  it  is  possible  to  determine  p  exactly.  If  we  let  Z  ■  X  . . /X  ,  we  see 

n  ih-i  n 

that  Z  »  p  when  I_  is  zero.  Because  P(I  -  0)«p  >  0,  I  will  be  zero  infi- 
n  »  n  n 

nltely  often.  By  waiting  for  the  first  repeated  value  of  Zn,  we  find 

that  p  is  equal  to  this  repeated  value.  The  quantity  p  will  also  be  the 

minimum  value  for  2.  because 

n 


We  note  that  Zn  has  a  continuous  distribution  when  W  1  and  has  probability 
p  concentrated  at  p»  So  tha  only  value  that  will  repeat  in  the  sequence 
is  p.  In  practice  other  values  will  have  a  small  probability  of  occurrence 
due  to  tha  discreteness  of  the  random  number  generator. 
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Gsver  sad  Lewi*  (Kef.  1)  point  out  that  if  we  use  the  stopping  tine 

T  ■  win  {  n:ZQ  repents  Its  previous  alnlaua  value} 

then  T  Is  the  sub  of  two  geoaetric  randan  variables  plus  one.  So  T  has 
expectation  1  +  (2/p)  and  variance  2(l-p)/p  .  Clearly  If  p  Is  not  too  snail 
the  expected  value  of  T  and  Its  variance  will  be  snail.  In  fact.  It  Is  easy 
to  determine  the  exact  probability  distribution  for  T, 

P(T-n)  -  (n-2)  p2  (l-p)n~3  for  n  >3  (3) 

■  0  for  n  <3. 

Fran  Eq.  (3)  It  Is  easy  to  determine  that 

P(T>u)  -  (l+(n-3)p  )(l-p)°'3  for  n  >4  (4) 

■  1  for  n  <3. 

So  as  long  as  p  Is  not  very  snail  It  Is  unlikely  that  T  will  be  very  large. 

On  the  other  hand.  If  there  Is  a  possibility  that  p  is  snail  and  one  cannot 
afford  to  take  more  than,  say,  Qq  samples,  we  would  recannend  using  the 
stopping  tine  T'  where 

T*  -  nln(T,  i^). 

When  T  >oq  the  logical  choice  for  an  estlnate  of  p  Is 

p  -  nln  {z  :  n  «  T’}. 
n 

The  estlnator  “p  Is  greater  than  or  equal  to  p  and  the  bias  will  be  snail  for 
reasonably  large  n. 

Lawrence  and  Lewis  (Kef.  6)  have  generalised  the  EAKMA  model  to  higher 
order  autoregressive  and  moving  average  terns.  In  particular,  they  define  the 
EAR(p)  processes  as  follows 
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11-1 


with  probability  a^ 
with  probability  a^ 


(5) 


°2Xi-2 

°  K 

p  i-p 


with  probability  a 


+  e. 


where 


al  "  °2^ 


P 

th  ‘j 


and 


°j)(1 


*  '  2 . »  *,  »  0 


for  lal,2,  ....  p 

and  Cj  has  the  distribution  required  eo  that  Xj  has  an  exponential  distribu¬ 
tion  with  paraaeter  1  for  each  1.  For  p  >  2  the  requlrenent  that  such  an 
1.1. d.  sequence  exists  laposes  additional  constraints  on  the  paraaaters. 
Lawrance  and  Lewis  derive  the  distribution  for  explicitly  only  In  the  case 

p-2. 

Caver  and  Lewis  (Bef.  1)  showed  that  for  the  EAR(l)  process,  once  p  has 
been  deteralned  through  the  sequential  estlaatlon  procedure,  the  Ej’s  can  be 
recovered  exactly  for  1>2.  Because  the  sequence  {  B  }  Is  1.1. d.  exponential 
with  the  rate  paraaeter  1,  the  usual  aaxlaua  likelihood  estlaates  for  X  can  be 
deteralned.  In  section  2  of  this  report,  it  Is  demonstrated  that  a  generali¬ 
sation  of  the  sequential  stopping  rule  can  be  used  to  explicitly  determine  the 
a^'s  for  each  1.  Section  3  discloses  how  a  conditional  likelihood  estimator 
can  be  deteralned  for  X.  For  p  >2  the  non-sero  e^'s  cannot  all  be  recovered 
and  hence  the  generalisation  of  the  result  for  p»l  Is  not  straightforward. 
Explicit  results  are  obtained  for  the  case  p«2. 


2.  DETERMINING  THE  AUTOREGRESSIVE  PARAMETERS 


Because  Cha  required  distribution  for  the  { en }  sequence  always  has 
positive  probability  concentrated  at  zero,  it  is  possible  to  determine 
a,,  a . a  by  keeping  track  of  the  ratios 

12  p 

X  X  X  X 

-a - ,  - - ,  -a - .  Once  the  value  of  = -  is  repeated,  that  repeated 

i-1  i-2  Ai-p  i-k 

Xi 

value  is  a  .  The  stopping  tine  T  is  then  the  smallest  n  such  that  = -  has 

*  i-k 

been  repeated  for  all  k  ■  1,2 . . 

For  the  case  when  p  -  2,  we  shall  determine  the  distribution  of  T,  its 
expectation  and  variance. 

The  EAR(2)  process  of  Lawrence  and  Lewis  is  given  as  follows: 


°i  Xl-1 
°2  Xi-2 


with  probability  1  -a^ 
with  probability  »2 


where 


ei  --I'i 


with  probability  Oj/O  -  <*2) 

with  probability  (1  -  o^)  (l  -  a2)/(l  -  6) 


1  SEj  with  probability  (1  -  <*2)  (c^  -  o2)  /{(l  +  Oj  I 

l  -  a2)(l  -  «)}  J 

and  fi  ■  (1  +  -  a2>  a2  and  {e^}  Is  an  1.1. d.  exponential  sequence  with 

parameter  1. 


and  T9  ■  min 


f  Xi  1 

Let  Tj  -  min  4  n:  ^ - is  the  same  for  two  values  of  1  <  n  land  T2  »  m 

•|n:  —  is  the  same  for  two  values  of  i  $  z|.  Then  let  T  •  max  {T^,^}. 


Now  Xj  •  with  probability  Pj  -  «j(l  -  o2)/(l  +  _  *i  " 

°2*i-2  Pr°hablllty  ?2  .  o^Oj/d  ♦  ~  «2^* 
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Me  consider  the  stochastic  sequence  {y^} ^  where  y^  -  0,  1  or  2.  The 
yj*s  ere  independent  random  vsrlebles  with  Pfy^  -  l]  -  p  t  p[y^  -2]  - 
end  Pfy^  ■  0]  •  1  »  P  -  P^.  Let  V  be  the  first  tiae  that  both  1  and  2  ere 
repeated.  Clearly  T  •  V  +  1. 

Slwple  combinatorial  arguments  show  that  p(v  ■  n  +  1]  -  p[t  ■  n  +  2]  ■ 
n  {pj  f Cl  -  Pj)""1  -  (1  -  Pj  -  Pj)""1]  +  Pj  [(1  "  P2)0_1  -  (1  -  Pj  -  Pj)"”1]} 
n(n  -  1)  PjP2  (Pj  +  P2)(l  -  Pi  -  P2)n"2  for  n  >  4.  E(T)  -  1  +  E(V)  and 
Var(T)  -  Var(V).  Computations  show 


E(T)  -  1  +  l-  +  1 


2P1P2 


2 


and 


Far(T)  - 


2P1P2 


~y  “  j-  *  “j“  “  T~  *  CP ¥  F ) - - - 1  + - - — j 

Pj  F1  P*  p2  iPi  *  V  (Pj  +  P2)z  (Pj  +  P2>3 


32  PI  P2 


‘'j'j 


»,  *  »,>T 


For  Bj  -  0.5  end  a2  *  0.4,  E(T)  -  13.86,  whereas  for  the  EAR(l)  process 
with  p  •  0.3  I(T)  "  5,  so  E(T)  increases  significantly  as  the  order  of  the 
process  increases.  In  prlnclpls  the  distribution  of  T  can  be  determined  for 
any  order  p  bat  apparently  the  distribution  becomes  more  complicated.  Clearly 
E(T)  grows  as  the  order  Is  increased  and  probably  Var(T)  also  grows  as  the 
order  is  increased.  For  higher  order  models  it  may  be  necessary  to  truncate 
the  stopping  time.  However,  it  is  not  deer  how  one  would  estimate  the 
Oj's  which  have  not  been  determined  by  repetition. 

The  gamma  first  order  autoregressive  process  of  Gaver  and  Lewis  (Ref.  1) 
(Gtt(l))  can  ba  generalised  to  higher  order  models  in  the  same  way  that 
Lswranca  and  Lewis  generalised  the  EAR(l)  process.  In  fact,  the  GAR(p) 
process  can  be  thought  of  ae  the  sum  of  k  EAR(p)  processes  when  the  parameter 
k  is  an  integer.  It  is 'obtained  in  the  following  way: 

Let  XQj,  X^,  ...,  X^  be  k  EAR(p)  processes  each  with  the 

earns  parameters  a  ,  a.,  ...,  a  ,  and  X.  The  processes  are  related  in  that 


lf  *nl  “  °rX(n-l)l  +  enl  then  Xn2  "  °rX(n-l)2  +  en2 . 

xnp  "  arX(n-l)p  +  enp  tae  r  "  *•  2*  •**»  p‘  ^  8e<*ueace8 

{e  {e  {e  }  are  independent.  Define 

ni  nz  np 


S  -  l  X 

*  1.1 


Then  SR  is  e  GAlt(p)  process  and  just  as  in  Eq.  (5)  we  have 


°i  si-i 
S1  ”  a2  Sl-2 


with  probability 

with  probability  a^  +  el 


a  S. 

P  1-P 


with  probability  a 


where 


-  (1  -  «2>.  ^  V  *,  -  **  <1  -  «t+l> 

t  »  2 . .  -  1,  1  ><Xj>0  for  1  -  1,  2 ,  •■>,  p 


*i  "  l  c11* 
1  J-l  13 


The  deterai nation  of  the  's  for  the  GAR(p)  process  is  the  saae  as  for 
the  EAR(p)  process.  Because  the  stopping  tlae  T  depends  only  on  the  sequence 

{y  }n  when  p  ■  2,  the  stopping  tlae  for  the  GAR( 2)  process  has  the  same  dis- 
1  1-1 

trlbution  as  for  the  EAR(2)  process. 


3.  ESTIMATION  of  lambda 


Once  Che  '■  have  been  determined  for  1*1,2 . .  m  can  compute  the 

following  residuals:  rn  -  Xj  -  c^Xj.j,  ri2  -  Xj  -  OjXj.j  ...  rJp  -  Xj  - 
OpXj_p.  When  rij  "  0  for  some  j  this  indicates  that  Zj  •  0  and  Xj  *  UjXj.j. 

He  can  then  determine  a  conditional  likelihood  for  the  residuals  given  the  eat 
of  ej  which  are  zero.  For  those  1  for  which  rj^  *  0  for  any  j,  £j  is  greater 
than  zero. 

We  shall  now  consider  the  case  p  -  2  for  simplicity.  Let  I|  - 
[1:  rjj  >  0  and  ri2  >  01 »  *2  “  ll!  rn  <  0  and  ri2  >  OJ  and  I3  * 

[1:  ru  >  0  and  t^2  <  0]  and  let 


I  -  U  I  . 
1-1  3 


Let  fc  be  the  number  of  elements  in  I.  When  ie^,  Cj  -  ti2  >  0,  and  when  1  el- 
el  •  >  0.  If  i£lj  we  do  not  know  whether  C|  ■  tjj  or  -  r^.  For  ICI2 


the  conditional  likelihood  la  le 


and  for  ielj  it  is  Xe 


.  When  1  el , 


Tly  know  that  >  0  and  is  either  r^j  or  r^* 


Consequently,  the  conditional  likelihood  is 


-Xr 

«2^e  +  °2e 


■>r’2]. 


Because  the  £j 's  are  independent  given  that  they  are  not  zero,  the  conditional 
likelihood  Le  is  given  by 

i*  •  »*  j.  f«  -  .'lr'2  .a. 


Maximizing  Lc  yields  s  conditional  maximum  likelihood  estimator  and  gener¬ 
alises  the  method  of  Gaver  and  Lewis.  In  general,  for  the  pth  order  process 
there  will  be  2P-1  sets  1^  to  consider  since  it  is  possible  for  r^t  to  be  less 
than  zero  or  greater  than  zero  for  each  i  and  t  but  r^(  cannot  be  less  than 
zero  for  i  fixed  and  each  t. 

Let 


and  k  be  the  number  of  elements  in  I.  Then  the  conditional  likelihood  is 


L 

c 


xk  Jx, 


-Xr 


J-l  Pij  * 


ij 


iSi,  j-l 


-Xr 


PiJ  6 


lj 


lit 


-Xr 


2P-1 


IP 


The  Pjj's  depend  on  which  r^'s  are  less  than  zero  and  in  particular  Pjj  -  0 
if  rtj  <0. 

Another  reasonable  way  to  estimate  X  when  p  ■  2  is  obtained  by  consider¬ 
ing  the  following  equation: 


X£  "  *°lTllXi-l  +  a2^  “  Xl-2^  “  6i  f°r  *  M  3,4 


n 


where 


PCY^  -  0)  ■  mg  »  1  -  P  (Tt  -  1)  and  the  sequence  {y^}  (7) 

is  1.1. d.  and  independent  of  Xj,  X2»  **.,  Xn_p 
Because  8(6^)  ■  1/X  and  E(yt)  ■  1  -  02, 

^XJ  *  °1  °  "  °2)  XJ-1  "  °2  Xj-2^ 


is  an  unbiased  and  consistent  estimate  for  1/X.  Consequently 


(®-3>/  ^  [Xj  -  aj  (1  -  o2)  Xj_j  -  <x\  X^2] 


will  be  a  consistent  estimator  for  X. 

For  the  GAR(p)  with  k  known,  the  conditional  likelihood  approach  could  be 

employed.  The  simple  approach  given  in  the  preceding  paragraph  can  also  be 

used.  E(e  )  -  k/X  and  so  the  estimator 
n 


k(n-3)/  I  {Sj  -  (1  -  «2)  Sj.j  -  <*2  sj_2> 

3“^ 


will  be  a  consistent  estimator  for  X. 
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4.  CONCLUSIONS 


\ 


For  low  order  EAR  or  GAR  models  this  sequential  parameter  estimation 
procedure  provides  a  satisfactory  way  of  determining  the  o^'s  and  then 
estimating  X.  Table  1  shows  results  of  simulating  the  process  for  various 
values  of  and  a^.  The  stopping  time  T  Is  replicated  fifty  times  and  the 
sample  mean  (T),  sample  variance  (S^)  are  compared  with  their  theoretical 
values  E(T)  and  V(T)  respectively.  Also,  the  maximum  value  of  T(TUX)  Is 
given  for  each  and  a^.  Even  for  small  values  of  the  's  it  may  be 
possible  to  use  the  stopping  time  because  T  is  unlikely  to  exceed  500. 
Considering  the  Xj's  to  be  the  Interarrival  times  for  a  point  process,  the  EAR 
process  Introduces  a  correlation  structure  to  the 'time  between  events  and 
hence  generalises  the  Poisson  process.  Generalizations  of  the  Poisson  process 
are  Important  because  they  allow  for  greater  flexibility  in  modelling  series 
of  failure  times.  These  andels  can  then  be  used  to  assess  the  risks 
associated  with  rare  catastrophic  events  such  as  an  accident  at  a  nuclear 
power  plant. 
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Table  1.  Simulation  of  the  EAR{2)  Process 


®i 

®2 

121  a. 

T 

V(T) 

S2 

T 

aax 

0.2 

0.1 

111.12 

113.64 

5865.9 

5932.9 

356 

0.2 

0.2 

51.90 

46.34 

1096.7 

702.8 

123 

0.2 

0.3 

32.95 

34.06 

320.5 

407.3 

85 

0.2 

0.4 

24.41 

22.58 

106.9 

195.0 

46 

0.4 

0.1 

66.07 

68.18 

2021.6 

2033.5 

232 

0.4 

0.2 

31.53 

29.56 

382.6 

363.0 

85 

0.4 

0.3 

20.53 

20.30 

112.1 

163.7 

64 

0.4 

0.4 

15.63 

14.60 

36.3 

36.7 

37 

0.5 

0.1 

57.14 

61.48 

1492.7 

1999.5 

232 

0.5 

0.2 

27.48 

27.96 

283.8 

342.0 

85 

0.5 

0.3 

.18.05 

,  18.80 

83.2 

140.4 

51 

0.5 

0.4, 

13*88 

13.00 

26.5 

27.1 

29 
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